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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 
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Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 Responsive to communication(s) filed on 5 April 2004 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
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6) IEl Claim(s) 35-52 is/are rejected. 
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Application Papers 
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a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received, 
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DETAILED ACTION 



1 . The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 



2. The reply filed 05 April 2004 was applied with the effect that claims 35-52 were 
elected and examined on the merits. 



3. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01 (o). Correction 
of the following is required: 

- There is no disclosure of the "syntactically different form" of claim 39 (2 nd line). 

- The Examiner is proceeding with the understanding that the claim is for producing 
the speech into a different form, as speech-to-text. 



Response to Amendment 



Specification 



4. The lengthy substitute specification has not been checked to the extent 
necessary to determine the presence of all possible minor errors. Applicant's 
cooperation is requested in correcting any errors of which applicant may become aware 
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in the specification, such as that the word "fort 1 which should be - - for - - (page 13, 2 nd 
line of the amendment, paper #5, to page 50 line 10) is as intended in the substitute 
specification. 

5. The specification is objected to as failing to provide proper antecedent basis for 
the claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01 (o). Correction 
of the following is required: 

- The feature of library offormants" for claims 44 & 45 is not found in the disclosure. 

- The feature of "abridging the content' for claim 47 is not found in the disclosure. The 
Examiner is proceeding with the understanding that the intent of the feature is to 

" prepare for publication" 

6. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

The following title is suggested: 

"Displaying the Source or Meaning of Sounds and Speech". 

Claim Rejections - 35 USC § 112 

7. The following is a quotation of the first paragraph of 35 U.S.C. 1 1 2: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 
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8. Claims 39 and 47 are rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter 
which was not described in the specification in such a way as to reasonably convey to 
one skilled in the relevant art that the inventor(s), at the time the application was filed, 
had possession of the claimed invention. 

- Regarding claim 39, in order for the "syntactically different" feature to take place 
some attribute such as number, sex or tense would change, such as from "past" to 
"future" tense. There is no disclosure that supports the limitation of "syntactically 
different form" of claim 39 (2 nd line). The Examiner is proceeding with the 
understanding that the claim is for producing the speech into a different "format, as 
speech-to-text. 

- Regarding claim 47, because the term "abridging a content may indicate the 
extremes of editing from shortening or elaborating, a conclusion is required that 
would leave the claim subject to interpretation. The Examiner is proceeding with the 
understanding that the claim is intended to "prepare for publication", i.e., to change 
from sound to visual formats. 

Claim Rejections - 35 USC § 102 

9. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 
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(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 



10. Claim 35, 36, 38, 39 and 41 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Marlev 813 (U.S. Patent 4,181,813 A). 

1 1 . Regarding claim 35, the invention of Marlev 813 for speech recognition reads on 
every feature of the claim for a prosthetic hearing device as follows: 

- Marlev 813 (32-48 in figure 3) reads on the feature of a sensor for detecting a 
linguistic sound; 

- Marlev 813 (250-256 in figure 9) reads on the feature of a sound recognition 
processor that performs sound recognition on the detected linguistic sound; 

- Marlev' 813 reads on the feature of a sound information generator that analyzes 
results of sound recognition by the sound recognition processor to comprehend a 
semantic meaning (column 1 line 65) in the linguistic sound and generates sound 
information (12' in figure 3) in which the linguistic sound is transformed so as to 
become assistive in understanding the semantic meaning (column 3 lines 4-7); and 

- Marlev 813 reads on the feature of an output device that outputs the sound 
information to a user (28 in figure 1 ). 

12. Regarding claims 36 and 38, the claims are set forth with the same limitations as 
claim 35. Marlev 813 reads on the feature where the sound recognition processor 
performs sound recognition (for claim 38, 'linguistic sound recognition" - see column 7 



States. 
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lines 7-10) in view of a physical state of the user (column 3 lines 57-60), an operating 
condition of the prosthetic hearing device or a purpose for use of the device by the user. 

13. Regarding claim 39 as understood by the Examiner, the claim is set forth with the 
same limitations as claim 35. Marley' 813 reads on the feature where the sound 
information generator transforms the linguistic sound into a syntactically different form 
(24^12 in figure 3, 24->28 in figure 1). 

14. Regarding claim 41 , the claim is set forth with the same limitations as claim 35. 
Marley 813 reads on the feature where the sound information generator reproduces 
sound information previously produced when it determines from results of speech 
recognition by the sound recognition processor that it is necessary to reproduce the 
previously produced sound information (claim 4 lines 27-28). 

Claim Rejections - 35 USC § 103 

Marley 813 & Watanabe et al 

15. Claim 37 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marley 813 in view of Watanabe et al (U.S. Patent 4,653,097 A). 

1 6. Regarding claim 37, the claim is set forth with the same limitations as claim 35. 
Marley 813 teaches the feature where the sound recognition processor performs speaker 
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recognition; speaker identification or speaker verification (column 1 lines 32-34) but fails 
to teach the feature of acknowledging speakers. 

Watanabe et aA with the invention for individual verification apparatus reads on 
the feature that the sound information generator generates sound information 
representing results of the recognition, identification and/or verification (claim 2 lines 32- 
34 - see S36 in figure 6). 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of 
Watanabe et al to the device/method of Marlev 813 so as to indicate that the 
authorization attempt was valid or unrecognized. 

Marlev 813 & Amano et al 

17. Claim 40 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marlev 813 in view of Amano et al (Japan Patent 02-014000 A). 

18. Regarding claim 37, the claim is set forth with the same limitations as claim 35. 
Marlev 813 does not mention adding modifiers to a language. Amano et ah with the 
invention for a voice-recognizing device, teaches the feature where the sound 
information generator transforms the linguistic sound by adding thereto a modifying 
language (with the post-processing of lines 3-4 of paragraph [57], Purpose. It would 
have been obvious to a person of ordinary skill in the art of speech signal processing at 
the time of the invention to apply the method/teachings of Amano et al to the 
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device/method of Marlev' 813 as a form of the well-known step of normalizing to complete 
abbreviations, define acronyms and finish uncompleted aphorisms. 

Marlev' 813 & Yamamoto 

19. Claims 42 and 43 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Marlev' 813 in view of Yamamoto (Japan Patent 58-062738). 

20. Regarding claim 42, the claim is set forth with the same limitations as claim 35. 
Marlev 813 does not mention repeating synthesized sound. Yamamoto , with the 
invention to understand a message by changing the speed of utterance of same 
messages repeatedly in response to the instruction of repetition by the operator reads 
on the feature where, when receiving a reproduction instruction through the input device 
from the user, reproduces sound information previously produced (last 2 lines of 
paragraph [57], constitution). It would have been obvious to a person of ordinary skill in 
the art of speech signal processing at the time of the invention to apply the method 
and/or teachings of Yamamoto to the device/method of Marlev 813 to insure that the 
listener is given every opportunity to receive and understand the information. 

21 . Regarding claim 43, the claim is set forth with the same limitations as claim 35. 
Marlev' 813 does not mention repeating synthesized sound at different speed. Yamamoto 
reads on the feature where the sound information generator controls an output rate of 
the sound information (1 st line of last 2 lines of paragraph [57], purpose). It would have 
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been obvious to a person of ordinary skill in the art of speech signal processing at the 
time of the invention to apply the method and/or teachings of Yamamoto to the 
device/method of Marlev 813 to insure that the repetition of "misheard" done in such a 
way to facilitate understanding rather than to limit time, the element of time having been 
made irrelevant due to the hearing error. 

Marlev' 813 & Coker etal 

22. Claims 44 and 45 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Marlev' 813 in view of Coker et al (U.S. Patent 3,704,345 A). 

23. Regarding claim 44 as understood by the Examiner, the claim is set forth with the 
same limitations as claim 35. Marlev 813 does not mention formants. Coker et ah with 
the invention for conversion of printed text into synthetic speech, reads on the feature 
where the output device outputs the sound information using a formant synthesized by 
the sound information generator (column 13 lines 24-29). 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method/teachings of Coker et 
alto the device/method of Marlev 813 so as to produce a quantifiable sound. 

24. Regarding claim 45 as understood by the Examiner, the claim is set forth with the 
same limitations as claim 35. Marlev 813 does not mention formants. Coker et a/ , with 
the invention for conversion of printed text into synthetic speech, reads on the feature of 
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a memory that has stored a library offormants, where the output device outputs the 
sound information using a formant selected by the sound information generator from the 
library (column 13 lines 27-29). 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method and/or teachings of 
Coker et al to the device and/or method of Marley 813 so as to save processing by 
eliminating redundant creating operations. 

Marley 813 & Takegawa 

25. Claims 46-47 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marley' 813 in view of Takegawa (Japan Patent 10-123450). 

26. Regarding claim 46, the claim is set forth with the same limitations as claim 35. 
Marley' 813 does not mention foreign language. Takegawa reads on the feature where 
the sound information generator generates the sound information in languages different 
from a language of the detected linguistic sound (paragraph [0003] line 3). It would 
have been obvious to a person of ordinary skill in the art of speech signal processing at 
the time of the invention to apply the method/teachings of Takegawa to the 
device/method of Marley 813 so as to provide a translation for one individual without 
disrupting others. 
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27. Regarding claim 47 as understood by the Examiner, the claim is set forth with the 
same limitations as claim 35. Marley 813 does not mention abridging. Takegawa , with 
the invention for a heads up display device with sound recognizing function for 
converting a sound signal collected by a microphone input, converted into a character 
signal (text data) and sent to an output terminal, converted into a picture by a liquid 
crystal display device and displaying the picture, reads on the feature where the sound 
information generator generates the sound information abridging a content of the 
linguistic sound (with converting, line 2 of the problems to be solved section (57) in the 
Abstract). 

It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method and/or teachings of 
Takegawa to the device/method of Marley 813 in order to visualize the source of a sound. 

Marley' 813 & Butnaru etal 

28. Claim 48 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marley 813 in view of Butnaru et al (U.S. Patent 6,240,392 B1 ). 

29. Regarding claim 48, the claim is set forth with the same limitations as claim 35. 
Marley 813 performs verbatim, which does not correlate to providing semantic meaning. 
Butnaru et a/ , with the invention for communication for deaf and mute persons, reads on 
the feature where the sound information generator displays on the display an image 
associated with the semantic meaning in the linguistic sound (lines 14-18 in the 
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Abstract). It would have been obvious to a person of ordinary skill in the art of speech 
signal processing at the time of the invention to apply the method and/or teachings of 
Butnaru et a/ to the device and/or method of Marlev 813 in order to visualize a sound. 

Marlev' 813 , Butnaru etaL Hoffberg eta/" 6 & Tsukahara 

30. Claim 49 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marlev 813 in view of Butnaru et al and further in view of Hoffberg et a/ 996 (U.S. Patent 
6,400,996 B1 ) and further in view of Tsukahara (U.S. Patent 6,460,056 B1 ). 

31 . Regarding claim 4, the claim is set forth with the same limitations as claim 48. 
Marlev 813 reads on the feature that selects and displays at least one image associated 
with the semantic meaning in the linguistic sound (10 in figure 1 ) but neither Marlev' 813 
nor Butnaru et al specify a multimedia library. 

Hoffberg et a/ 996 , with the invention for adaptive pattern recognition based 
control, teaches the features of a library of images comprising still and motion pictures, 
(column 42 line 7 - see column 124 lines 45-67)), symbols (column 74 line 11), 
characters, (column 124 line 49), notes, (as videotext, column 124 lines 49 & 58), 
photos (as frame in column 125 line 3), animations, (column 102 line 23), illustrations, 
(to enhance, column 124 line 59 - see column 125 lines 5-7) and colors, (column 125 
line 8) but does not mention spectral patterns. 

Tsukahara et ai with the method of analyzing input speech and speech analysis 
apparatus, reads on the feature of voice spectrum patterns (column 3 lines 2-5). ... It 
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would have been obvious to a person of ordinary skill in the art of speech signal 
processing at the time of the invention to apply the method/teachings of Hoffberq et 
a/ 996 & Tsukahara to the device/method of Marlev' 813 & Butnaru et al to provide a 
desired user function from a single source. 

Marlev' 813 , Butnaru et al & Horii 

32. Claim 50 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marlev 813 in view of Butnaru et al and further in view of Horii (U.S. Patent 6,460,056). 

33. Regarding claim 50, the claim is set forth with the same limitations as claim 35. 
Marlev 813 does not mention sign language while Butnaru et al obviates it. Horii , with the 
invention for displaying sign language images corresponding to input information, reads 
on the feature where the sound information generator translates the linguistic sound into 
a sign language representing the semantic meaning in the linguistic sound and displays 
the sign language (Abstract lines 5-9). It would have been obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method and/or teachings of Horii to the device and/or method of Marlev 813 & 
Butnaru et al in order to display images corresponding to a document. 

Marlev 813 & Hoffberq et a/' 996 

34. Claim 51 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marlev 813 in view of Hoffberq et a/' 996 . 
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35. Regarding claim 51 , the claim is set forth with the same limitations as claim 35. 
Marlev 813 does not indicate more than one source of input Hoffberq et a/ 996 reads on 
the feature where the sensor selectively detects a linguistic sound from a specific sound 
sources (column 61 lines 39-45). It would have been obvious to a person of ordinary 
skill in the art of speech signal processing at the time of the invention to apply the 
method/teachings of Hoffberq et a/ 996 to the device/method of Marlev 813 to resolve the 
well-known "cocktail party" environmental situation of concentrating on one audio signal 
from many in a noisy environment. 

Marlev' 813 & Abe 705 

36. Claim 52 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Marlev 813 in view of Abe et a/ 705 . 



37. Regarding claim 52, the claim is set forth with the same limitations as claim 35. 
Marlev 813 does not employ a video camera. Abe et a/ 705 , with the invention for sign 
language translation, reads on the feature of further comprising a video camera (column 
4 line 21 ), where the sound information generator interprets a sign language captured 
by the camera and generates sound information representing the interpreted sign 
language (column 9 lines 28-30 & 37-39). It would have been obvious to a person of 
ordinary skill in the art of speech signal processing at the time of the invention to apply 
the method/teachings of Abe et a/ 705 to the device/method of Marlev 813 , employing 
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existing security equipment as assistive technology to provide alternative access for 
individuals with disabilities, thus avoiding a separate keyboard component. 



Conclusion 

38. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

- Hoffberq et a/ 145 (U.S. Patent 6,640,145 B2) media recording device with packet 
data interface. 

- Tsianq (U.S. Patent 5,377,302 A) system for recognizing speech. 

- Belleqarda et al (U.S. Patent 5,502,774 A) automatic recognition of a consistent 
message using multiple complimentary sources of information. 

- Abe et al 050 (U.S. Patent 5,544,050 A) sign language learning system and method. 

- Okada et al (Japan Patent 02-01 1438) device for reading information outside vehicle 
by recognizing an object, extracting the recognition signals, and distinguishing the 
existence of the approaching object from the compared result and giving warning. 
An optical recognizing recognizes an object by means of light, sound waves, or 
electromagnetic waves, etc. and which outputs the image of the object. 

- Numata et al (Japan Patent 2003-044497) mobile picture book for telling a name or 
the like for the image or sound of an object. 

- Tsurata (Japan Patent 04-075098) position of voice recognition 
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- Dutta et al (U.S. Patent 6,453,294 B1 ) dynamic destination-determined multimedia 
avatars for interactive on-line communications. 

- Preston (U.S. Patent 3,636,261 A) method and apparatus for optical speech 
correlation. 

- Hatazaki (U.S. Patent 5,671 ,329 A) speech dialogue system in which recognition 
and understanding process, application process, and voice input response are 
performed simultaneously with voice input. 

- Marlev' 846 (U.S. Patent 4,284,846 A) System and method for sound recognition. 

- Belleqarda et al 774 (U.S. Patent 5,502,774 A) automatic recognition of a consistent 
message using multiple complimentary sources of information. 

- Belleqarda et a/ 809 (U.S. Patent 5,621 ,809 A) computer program product for 
automatic recognition of a consistent message using multiple complimentary 
sources of information. 

39. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Daniel A. Nolan at telephone (703) 305-1368 whose 
normal business hours are Mon, Tue, Thu & Fri, from 7 AM to 5 PM. 

If attempts to contact the examiner by telephone are unsuccessful, supervisor 
Richemond Dorvil can be reached at (703)305-9645. 

The fax phone number for Technology Center 2600 is (703)872-9314. Label 
informal and draft communications as "DRAFT" or "PROPOSED", & designate formal 
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communications as "EXPEDITED PROCEDURE". Formal response to this action may 
be faxed according to the above instructions, 

or mailed to: 



or hand-deliver to: Crystal Park 2, 

2121 Crystal Drive, Arlington, VA, 
Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to Technology Center 2600 Customer Service Office at 
telephone number (703) 306-0377. 



P.O. Box 1450 
Alexandria, VA 22313-1450 



Daniel A. Nolan 
Examiner 
Art Unit 2654 



DAN/d 

May 17, 2004 




